01.02 Comprehensions

Using comprehensions is often a way both to make code more compact and to shift our focus from the "how" to the "what". It is an expression that uses the same keywords as loop and conditional blocks, but inverts their order to focus on the data rather than on the procedure.

Simply changing the form of expression can often make a surprisingly large difference in how we reason about code and how easy it is to understand. The ternary operator also performs a similar restructuring of our focus, using the same keywords in a different order.

List Comprehensions

A way to create a new list from existing list based on defined logic

Unconditional Compreshensions


In [ ]:
# Original
doubled_numbers = []
for n in range(1,6):
    doubled_numbers.append(n*2)

print(doubled_numbers)

In [ ]:
#list compreshensions
doubled_numbers = [n * 2 for n in range(1,12,2)] # 1 ,3, 5, 7, 9, 11
print(doubled_numbers)

Conditional Compreshensions


In [ ]:
doubled_odds = []

for n in range(1,12):
    if n % 2 == 1:
        doubled_odds.append(n * 2)
print(doubled_odds)

In [ ]:
doubled_odds = [n * 2 for n in range(1,12) if n% 2 == 1]

print(doubled_odds)

!!!! Tip !!!!

  • Copy the variable assignment for our new empty list (line 3)
  • Copy the expression that we’ve been append-ing into this new list (line 6)
  • Copy the for loop line, excluding the final : (line 4)
  • Copy the if statement line, also without the : (line 5)

In [ ]:
# FROM
numbers = range(2,10)

doubled_odds = []
for n in numbers:
    if n % 2 == 1:
        doubled_odds.append(n * 2)
print(doubled_odds)

In [ ]:
# TO
numbers = range(2,10)

doubled_odds = [n * 2 for n in numbers if n % 2 == 1]

Nested if statements in for loop


In [ ]:
l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0]

lst = []
for v in l:
    if v == 0 :
        lst.append ('Zero')
    else:
        if v % 2 == 0:
            lst.append  ('even')
        else:
            lst.append  ('odd')
            
print(lst)

In [ ]:
lst = ["zero" if v == 0 else "even" if v%2 == 0 else "odd" for v in l]
print(lst)

In [ ]:
print(['yes' if v == 1 else 'no' if v == 2 else 'idle' for v in l])

In [29]:
def flatten_list_new(lst, result=None):
    """Flattens a nested list
        >>> flatten_list([ [1, 2, [3, 4] ], [5, 6], 7])
        [1, 2, 3, 4, 5, 6, 7]
    """
    if result is None:
        result = []
#     else:
#     result = [x if not isinstance(x, list) else flatten_list_new(x, list) for x in lst]
#     result = [ x  if not isinstance(x, list) else isinstance(x, list) for x in lst ]
    result = [x for x in lst if not isinstance(x, list) else isinstance(x, list)]
#     for x in lst:
#         if isinstance(x, list):
#             flatten_list_new(x, result)
#         else:
#             result.append(x)

    return result
lst = [ [1, 2, [3, 4] ], [5, 6], 7]
  
print(flatten_list_new(lst))


  File "<ipython-input-29-31b81cb9a624>", line 11
    result = [x for x in lst if not isinstance(x, list) else isinstance(x, list)]
                                                           ^
SyntaxError: invalid syntax

In [21]:
lst = []
for a in range(10):
    if a % 2==0:
        for x in range(a, 10):
            lst.append(x)

print(lst)


[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 2, 3, 4, 5, 6, 7, 8, 9, 4, 5, 6, 7, 8, 9, 6, 7, 8, 9, 8, 9]

In [ ]:
n = 10
lsts = [x for a in range(n) if a % 2 == 0 for x in range(a, 10)]

print(lsts)

In [19]:
%%time
import os

files = []

for d in os.walk(r"E:\code\mj\lep\Section 1 - Core Python"):
    for f in d[2]:
        if f.endswith(".py"):
            files.append(os.path.join(d[0], f))

print(len(files))


134
Wall time: 11.2 ms

In [20]:
%%time

import os

files = [os.path.join(d[0], f) 
            for d in os.walk(r"E:\code\mj\lep\Section 1 - Core Python")
                 for f in d[2] if f.endswith(".py")]

print(len(files))


134
Wall time: 8.61 ms

In [4]:
%%time

import os

files = [os.path.join(d[0], f) 
            for d in os.walk(r"E:\code\mj\lep\Section 1 - Core Python")
                 for f in d[2]  if f.endswith(".py")]

print(len(files))


134
Wall time: 7.75 ms

In [9]:
%%time
restFiles = []

for d in os.walk(r"C:\apps"):
    if "etc" in d[0]:
        for f in d[2]:
            if f.endswith(".txt"):
                restFiles.append(os.path.join(d[0], f))
print(len(restFiles))


0
Wall time: 977 µs

In [10]:
%%time
restFiles = [os.path.join(d[0], f) 
                 for d in os.walk(r"C:\apps") 
                     if "etc" in d[0]
                         for f in d[2] 
                             if f.endswith(".txt")]
print(len(restFiles))


0
Wall time: 977 µs

In [ ]:
%%time

matrix = []
for row_idx in range(0, 3):
    itmList = []
    for item_idx in range(0, 3):
        if item_idx == row_idx:
            itmList.append(1)
        else:
            itmList.append(0)
    matrix.append(itmList)
print(matrix)

In [ ]:
matrix = [ [ 1 if item_idx == row_idx 
               else 0 for item_idx in range(0, 3)] 
          for row_idx in range(0, 3) ]
print(matrix)

In [ ]:
lst = [1,2,34,4,5]
print(lst)
lst.append(2)
print(lst)

In [ ]:
lst.append(2)
print(lst)
l = set(lst)
print(l)

Set Comprehensions

Set comprehensions allow sets to be constructed using the same principles as list comprehensions, the only difference is that resulting sequence is a set and "{}" are used instead of "[]".


In [ ]:
names = [ 'aaLok', 'Manish', 'AalOK', 'Manish', 'Gupta', 'Johri', 'Mayank' ]

new_names1 = [name[0].upper() + name[1:].lower() for name in names if len(name) > 1 ]
new_names = sorted({name[0].upper() + name[1:].lower() for name in names if len(name) > 1 })
print(new_names1)
print(new_names)

Dictionary Comprehensions


In [ ]:
original = {'a':10, 'b': 34, 'A': 7, 'Z':3, "z": 199}

mcase_frequency = { k.lower() : original.get(k.lower(), 0) + original.get(k.upper(), 0) for k in original.keys() }
print(mcase_frequency)

In [ ]:
original = {'a':10, 'b': 34, 'A': 7, 'Z':3, "z": 199, 'c': 10}
flipped = {value: key for key, value in original.items()}
print(flipped)

In [ ]:
original = {'a':10, 'b': 34, 'A': 7, 'Z':3, "z": 199, 'c': 10}
newdict = {}
for key, value in original.items():
#     print(ori)
    if (value not in newdict):
        newdict[value] = key
print(newdict)

In [ ]:
newdict = {value: key for key, value in original.items() if (value not in newdict)}
print(newdict)

In [ ]:
x = {"a": 10, "b": 20, "c": 20}

print(x)
x["a"] = 100

print(x)

This map doesn’t take a named function. It takes an anonymous, inlined function defined with lambda. The parameters of the lambda are defined to the left of the colon. The function body is defined to the right of the colon. The result of running the function body is (implicitly) returned.

The unfunctional code below takes a list of real names and appends them with randomly assigned code names.


In [ ]:
import random

names_dict = {}
names = ["Mayank", "Manish", "Aalok", "Roshan Musheer"]
code_names = ['Mr. Normal', 'Mr. God', 'Mr. Cool', 'The Big Boss']

random.shuffle(code_names)

for i in range(len(names)):
    names_dict[names[i]] = code_names[i] 
        
print(names_dict)

In [ ]:
import random

names_dict = {}
names = ["Mayank", "Manish", "Aalok", "Roshan Musheer"]
code_names = ['Mr. Normal', 'Mr. God', 'Mr. Cool', 'The Big Boss']

random.shuffle(code_names)

names_dict = dict(zip(names, code_names))

print(names_dict)

In [ ]:
ld = [{'a': 10, 'b': 20}, {'p': 10, 'u': 100}]
dict([kv for d in ld for kv in d.items()])

Generator Comprehension

They are simply a generator expression with a parenthesis "()" around it. Otherwise, the syntax and the way of working is like list comprehension, but a generator comprehension returns a generator instead of a list.


In [ ]:
x = (x**2 for x in range(20))
print(x)
print(list(x))

In [ ]:
itm = 10
print(itm / 2)

Summary

When struggling to write a comprehension, don’t panic. Start with a for loop first and copy-paste your way into a comprehension.

Any for loop that looks like this:


In [ ]:
def condition_based_on(itm):
    return itm % 2 == 0

old_things = range(2,20, 3)
new_things = []
for ITEM in old_things:
    if condition_based_on(ITEM):
        new_things.append(ITEM)
print(new_things)

Can be rewritten into a list comprehension like this:


In [ ]:
new_things = [ITEM for ITEM in old_things if condition_based_on(ITEM)]
print(new_things)

NOTE

If you can nudge a for loop until it looks like the ones above, you can rewrite it as a list comprehension.